A randomized trial looking at planning prompts to reduce opioid prescribing

Prior work has demonstrated that personalized letters are effective at reducing opioid and benzodiazepine prescribing, but it is unclear whether If/when-then planning prompts would enhance this effect. We conducted a decedent-clustered trial which randomized 541 clinicians in Los Angeles County to receive a standard (n = 284), or comparator (n = 257) version of a letter with If/when-then prompts. We found a significant 12.85% (6.83%, 18.49%) and 8.32% (2.34%, 13.93%) decrease in the primary outcomes morphine (MME) and diazepam milligram equivalents (DME), respectively. This study confirms the benefit of planning prompts, and repeat letter exposure among clinicians with poor patient outcomes. Limitations include lack of generalizability and small sample size. Clinicaltrials.gov registration: NCT03856593.

Prior work has demonstrated that personalized letters are effective at reducing opioid and benzodiazepine prescribing, but it is unclear whether If/when-then planning prompts would enhance this effect.We conducted a decedentclustered trial which randomized 541 clinicians in Los Angeles County to receive a standard (n = 284), or comparator (n = 257) version of a letter with If/ when-then prompts.We found a significant 12.85% (6.83%, 18.49%) and 8.32% (2.34%, 13.93%) decrease in the primary outcomes morphine (MME) and diazepam milligram equivalents (DME), respectively.This study confirms the benefit of planning prompts, and repeat letter exposure among clinicians with poor patient outcomes.Limitations include lack of generalizability and small sample size.Clinicaltrials.govregistration: NCT03856593.
Despite a national effort to curb opioid addiction and fatal overdoses, excessive opioid prescribing continues to vary substantially over welldefined geographic areas 1 .One promising way to attack this problem involves notifying physicians by mail when one of their patients has a fatal opioid overdose and urging them to reduce opioid prescriptions to safeguard other patients.For example, an initial notification study found the intervention led to an approximately 10% reduction in morphine milligram equivalents (MMEs) dispensed, fewer new patients started on opioids, and a decrease in high-dose opioid prescriptions 1 to 4 months after receipt of the letter 2 .A follow-up study reported positive spillover effects on benzodiazepine prescription practices 3 , a demonstration that the letters improved safe prescription practices for a broader range of scheduled drugs.A separate study showed the effects of these notifications on prescription practices persisted for up to 1 year 4 .Notifications may have a lasting impact on patient safety and potentially could reduce the risk of future overdose.Overall, these studies provide evidence that notification of patients' fatal overdose, a low-cost solution that is easily scalable, may have several positive effects.
There might be ways to improve such letters to ensure robust, safe prescription practices.One potential approach is to incorporate planning prompts-sometimes referred to as implementation intentions or If/when-then plans-into these letters [4][5][6] .Planning prompts are mental rules that describe events meant to bring about concrete actions in specific situations: If (or when) situation S occurs, then engage in behavior B. For example, physicians who receive a notification of a patient's fatal overdose would be prompted to carry out a concrete plan of action that would be triggered by a specific set of circumstances (e.g., when their patients present with pain).
With the incorporation of planning prompts into these letters, physicians may be more likely to take concrete steps to modify their prescription practices and to reduce the risk of future overdose.For example, a physician might better use the information in the letter, if the letter guidance urges them to implement steps at the visit, such as discussing alternative pain management strategies or consulting with a pain management or addiction specialist for evaluation and care.
We hypothesize that the incorporation of planning prompts into notifications will further enhance the effectiveness of the communication, lead to even lower doses and fewer opioid orders, and possibly reduce overdose risk.
To test this hypothesis, we conducted a decedent-clustered randomized trial to compare the effectiveness of standard notification letters to those that incorporate If/when-then plans in Los Angeles County, where the standard letter is mandated by the County Board of Supervisors.We conjecture that If/when-then plain text in the letters will bridge the gap between physicians' intentions and actions; this will result in improved prescription practices.The trial's comparison of the effectiveness of a standard notification letter to one that incorporates If/when-then plans will provide insight into not only which letter leads to reduced dose and frequency of opioid orders but will also identify whether certain physicians, such as those who have more patient deaths, benefit the most from the more effective letter.Our trial aims to advance the design of effective interventions to improve patient safety and reduce the risk of future overdose.

Opioids sample
Figure 1 shows the sample progression.The Los Angeles Medical Examiner-Coroner examined 316 fatal accidental opioid-related overdoses from late October 2018 to late May 2020 in Los Angeles County.Of these, 236 decedents (74.7%) received at least one CURESdocumented scheduled drug from 541 prescribers in the 12 months prior to their death.Thirty-one prescribers (5.73%) had no opioid prescription during the study period and were removed.Three decedents (1.27%) received Scheduled II-IV prescriptions from these prescribers only and were also removed.Since we assigned each prescriber to a cluster by his or her first decedent, this resulted in a final analytic sample of 219 first-decedents and 510 clinicians.
There was no difference in decedent age, sex, race, or cause of death between study arms (Table 1).There was a small difference (P = 0.025) in prescriber professional practice between study arms (Table 2).On average, decedents received prescriptions from 2.40 (σ = ± 2.25) prescribers, with a range of 1-13.Four hundred sixty-nine prescribers (91.96%) had one decedent.Forty-one prescribers (8.04%) had more than one decedent and received a letter for each.The average number of decedents per prescriber (corresponding to letters sent) was 1.20 (σ = ± 0.40).The number of prescribers who had more than one decedent did not differ by study arm in the analytic sample (χ 2 = 1.21,P = 0.271).
To evaluate if outlier clinicians played a role in study effects, Fig. 2 shows the clinician-level total log MME distribution pre-to-post intervention between study arms among clinicians with one decedent and more than one decedent.Pre-intervention, the dependent variable did not differ by study arm (t = 0.83, P = 0.406), or among clinicians with one versus multiple decedents (t = −0.61,P = 0.544).The number of outliers also did not differ by study arm (χ 2 = 1.42,P = 0.233) or number of decedents (χ 2 = 0.48, P = 0.759) (Supplementary Table S2a).

Post-hoc morphine milligram equivalent analyses
There was no difference in the study-start and study-end coefficients (β = −0.06[95% CI: −0.16, 0.04]; P = 0.224), indicating that the effect persisted over time.There was a significant three-way interaction between study arm, time, and number of decedents, translating to a 31.41%([95% CI: 11.38%, 46.91%]; P = 0.004) greater decrease in total weekly MME for prescribers who received multiple comparator letters (Supplementary Table S3a).

Benzodiazepine sample
Thirteen clinicians (2.40%) did not prescribe a benzodiazepine during the study period and were removed from the analysis (Fig. 3).One decedent (0.42%) received scheduled prescriptions from these prescribers and was also removed.The final analytic sample was 528 clinicians and 220 first-decedents.
To evaluate if outliers contributed to the effect, Fig. 4 shows the pre-to-post intervention clinician-level logarithm transformed DME distribution by study arm and number of decedents.There were no differences by study arm in log DME (t = 1.50, P = 0.133).Preintervention log DME was slightly higher among clinicians with more than one decedent (t = −5.28,P < 0.001).The number of outliers also did not differ by study arm (χ 2 = 3.65, P = 0.125) or number of decedents (χ 2 = 0.09, P = 1.0) (Supplementary Table S2b).

Discussion
We sought to determine whether the addition of If/when-then plans could bridge the gap between physicians' intentions and actions in notifications of a fatal overdose in a physician's practice.Our findings suggest that If/when-then plans may reduce opioid prescription intensity and frequency, thereby reducing future risk.A moderator analysis showed that the If/when-then plan was more effective relative to the standard letter for prescribers who received multiple letters.These were persons with multiple deaths in their practice.There was also evidence of spillover to more judicious benzodiazepine prescribing.A safety analysis showed that study arms did not differ in the number of clinicians demonstrating a 20% reduction in filled benzodiazepine prescriptions.The If/when-then plan may have helped prescribers with complex or difficult patients to implement skills that promote safe prescription practices.Alternatively, the prescribers with multiple deaths may have lacked skills and benefited from a letter that carefully outlined steps to improve prescription safety.
The intervention is scalable.Forty-nine of the 50 U.S. states operate prescription drug monitoring programs, and every county in the U.S. has a medical examiner or coroner.These interventions could Predicted MME using coefficients from censored, mixed linear model (Table S1a) testing the two-sided hypothesis that change in pre-to-post MME does not differ by study arm.easily operate out of the medical examiner's office almost anywhere in the U.S.This intervention has the benefit of helping clinicians learn of deaths in their practice that they might not otherwise learn about.This creates a more balanced sample from which they can draw inferences about their patients' outcomes.The intervention is low-cost and requires only minimal changes to routine administrative tasks in the medical examiner's office.A downside to the intervention is that it cannot reach all clinicians, but only those with a death in their practice.
Those clinicians with a death in their practice are, however, in greatest need of receiving the intervention.Doctor et al. 2 compared a letter notifying clinicians of a death in their practice to a no-treatment control group.Morphine milligram equivalents in prescriptions filled by patients of letter recipients versus no intervention controls decreased by 9.7% (95% CI: 6.2 to 13.2%; P < 0.001) 2  From the standpoint of psychological mechanisms, the plan may have provided a simple way to take contingent action at pain-related visits.Our results are in line with previous research that shows If/when-then plans are effective in other environments [7][8][9] .
This study has several strengths.It is a randomized concealment design 10 that avoids standard pitfalls that come with trial enrollment, attrition, and non-response.The study also used unobtrusive measures (California's Controlled Substance Utilization Review and Evaluation System 2.0 [CURES]), which lowered the potential for reactivity.Despite the strengths of our study, there are several limitations to consider.First, our study was conducted in Los Angeles County, and while it is the largest County in the United States, the results may not be generalizable to other populations or geographic regions.Additionally, our sample size was limited, and our study only examined the   short-term effects of the intervention.Finally, we were unable to examine the effects of the intervention on patient outcomes, such as overdose rates or other adverse events.
In conclusion, our study provides evidence that If/when-then plans as additions to fatal overdose notification letters moderate physician prescription practices.Further research is needed to confirm the results, examine the long-term effects of the intervention, and explore its potential effects on patient outcomes.

Methods
The trial protocol contains details regarding the intervention, power calculations, randomization, inclusion criteria, and statistical approach (see Supplement).All study procedures were ethically compliant and reviewed and approved by the University of Southern California's Institutional Review Board (IRB) prior to trial implementation (UP-19-00172).As part of this approval, we were granted a waiver of informed consent and a full waiver of HIPAA authorization.Participants were not compensated.

Intervention
This cluster randomized controlled trial compares two versions of a personal notification of a patient's fatal overdose designed to reduce Schedule II-IV prescriptions and opioid dosages.Trial participants were II-IV prescribers to individuals who died of an overdose between late October 18, 2018 and late May 21, 2020 in Los Angeles County, where opioids were a contributing or primary cause.We received decedent demographics and toxicology information from the Los Angeles County Department of Medical Examiner-Coroner.The contributing cause of death was determined by the Medical Examiner's judgment.The decedent's gender was obtained from a governmentissued ID and, therefore, self-reported.Clinician gender was also selfreported, and retrieved via license search on the Department of Consumer Affairs site 11 .Gender was not considered in the study design.We received a waiver of consent, which does not allow us to report disaggregated, individual-level data for gender, or any other demographic characteristic.The Medical Examiner staff downloaded prescriber names for each decedent through the State of California Department of Justice's CURES online portal 12 ; data was maintained on a secure, electronic database with restricted access by the Los Angeles Department of Medical Examiner-Coroner.Prescribers were grouped into decedent clusters and randomized to receive a standard letter or comparator.
The standard letter (see Supplementary Methods S1) was used in Doctor et al. 2 .Both letters informed prescribers of their patient's opioid-related overdose, provided guidance on judicious prescription practices, and directed the prescriber to CURES.Both letters were also signed by the Los Angeles County Chief Medical Examiner-Coroner and one of the Los Angeles, Pasadena, or Long Beach Public Health Officers as determined by the prescriber's practice location.The comparator letter, however, included an If/when-then plan that advised the clinician to keep the letter's recommendations close when their next patient presents with pain (see Supplementary Methods S2).The If/when-then plan specifies a contingent action.Letters were sent to prescribers on a monthly basis between April 4, 2019 and July 8, 2020.
California pharmacies that dispense controlled substances are legally required to submit weekly reports to CURES.We accessed this data in collaboration with the research unit and program manager of CURES; per decedent, we notified administrators that clinician prescriptions would be prospectively analyzed post-randomization and letter receipt.We provided CURES with prescriber names to retrieve all Schedule II-IV prescriptions from October 1, 2017 to August 31, 2021.CURES replaced personal identifiers with randomized digit IDs for transfer to the University of Southern California.

Power calculations
Power was calculated with the statistical computing language R clus-terPower package.We assumed a mean of 5.5 prescribers per decedent 2 , and a coefficient of variation of 1.22.This limited the number of prescribers per decedent at the 99th percentile to 20.Relative to the intraclass correlation of 0.05-0.15 13found for most clinician process measures, we used a more conservative estimate of 0.20.Using the Taylor method for variance inflation due to unequal clusters and given a two-tailed test with a 5% Type 1 error rate, we had 80% power to detect a 50% reduction in mean MME or DME (σ = ± 140) with 103 decedents per study arm.

Inclusion and exclusion criteria
Decedents.Decedents were included in the randomization if they died of an opioid-related overdose in Los Angeles County from October 18, 2018 to May 21, 2020, and received a legitimate Schedule II-IV prescription from a clinician verified by CURES within the 12 months prior to their death.
Prescribers.Prescribers were included if they (1) were located and practicing in LA County, (2) had scheduled drug prescribing privileges and, (3) prescribed a Schedule II-IV drug within the 12 months prior to a patient's death where opioids were the primary or a contributing cause.

Randomization
Decedents who died of an opioid-related overdose in Los Angeles County from October 18, 2018 to May 21, 2020, and received a CURESverified Schedule II-IV prescription, were randomized to the comparator or standard letter.Decedent randomization was stratified by whether the decedent was prescribed a benzodiazepine in the year prior to death.For each of the 2 combinations of stratification levels, the principal investigator and lead analyst created a list of decedents.We used random.org'slist randomizer to randomize decedent clusters to a numbered list.The first half of the list was randomized to the standard arm, and the second half of the list was randomized to the comparator arm.If the number of decedents was not divisible by 2, the last decedents on the randomly numbered list were assigned by a ½ probability lottery on random.org.The randomization process was repeated for new prescribers each monthly cycle.Prescribers with multiple decedents received multiple letters.Their experimental condition did not change after the first letter, ensuring they received the same letter repeatedly.

Measures
In addition to opioids, we analyzed benzodiazepines for potential spillover effects.Opioid and benzodiazepine prescriptions dispensed in the 52 weeks prior to a prescriber's letter sent date were included in the pre-intervention period.Opioid and benzodiazepine prescriptions dispensed 4-52 weeks after the letter sent date were included in the post-intervention period.The first 4 weeks of follow-up were washed out to avoid contamination from undispensed prescriptions ordered before letter receipt.In the case of prescribers with multiple decedents, the first decedent's letter sent date defined the pre-and postintervention periods.

Outcomes
We used the Centers for Disease Control for MME conversion factors and guidelines (Supplemental Table S4a) 14 .We converted benzodiazepines to diazepam milligram equivalents (DME) based on information provided by Borrelli et al. 15 .(Supplemental Table S4b).Our primary outcomes were the change in total weekly MME and DME dispensed pre-to post-intervention between study arms.Perprescription average daily MME and DME were calculated by multiplying the prescription strength by the number of units prescribed per day and the conversion factor (i.e., strength*(quantity/days)*conversion factor) 15 .This was summed per clinician per week.We log-transformed total weekly MME and DME to ensure normality.We confirmed normality with quantile-quantile (q-q) plots.Preintervention outliers were detected using Tukey's fences 16 , which is 1.5 times the interquartile range plus or minus quartiles three and one, respectively.
Our exploratory outcomes were the difference in the likelihood of an opioid prescription greater than or equal to 50 MME and the difference in the likelihood of an opioid prescription greater than 90 MME.We also tested the difference in the probability of a new patient receiving an opioid or benzodiazepine prescription pre-to postintervention between study arms.To test patient safety associated with spillover effects, we assessed pre-to-post-intervention mean DME drop-offs greater than 20%.

Statistical analysis
Statistical analyses were executed using SAS version 9.4, STATA software version 16, and R version 4.3.2 17.We used a multilevel regression with left censoring to account for observations without any opioid or benzodiazepine prescriptions 18 .We regressed log MME and DME on the study arm (comparator vs. standard letter), the study period (preintervention vs. post-intervention), and the interaction between the study arm and the study period.To control for differences in decedent and prescriber behavior, we included a random intercept for the prescriber nested within the decedent.Model 1 is the effect of time (β 1 ) study arm (β 2 ), and the interaction between time and study arm (β 3 ) on Y for the ith prescriber and jth prescription, where Y is uncensored, latent weekly log MME or log DME.The coefficients are a [100*(1 − exp(β)] change in uncensored MME and DME per-level increase.δ i k ð Þ is the random intercept for prescriber nested within the decedent, with mean 0 and variance σi k ð Þ. Due to prescriber outliers in the standard letter group, we calculated trimmed preintervention means for Table 3 and Table 4 that resulted in the smallest difference between letter groups, which was 9% and 7% for opioid and benzodiazepine prescribers, respectively.We calculated adjusted post-intervention means to obtain the difference-in-difference in preto post-weekly MME and DME between the standard and comparator study arms.95% confidence intervals were bootstrapped using 2000 repetitions of 12,000 randomly selected observations.log ð ŶÞ We evaluated exploratory outcomes using mixed-effects, logistic regressions to test whether patients with prescribers randomized to the comparator letter were less likely to receive an opioid prescription greater than or equal to 50 MME or greater than 90 MME postintervention and the probability of a new patient receiving an opioid or benzodiazepine prescription post-intervention.We calculated the perclinician percent change in mean DME pre-to-post intervention and used logistic regression to test whether greater than 20% drop-offs were higher among clinicians in the comparator arm, controlling for the proportion of new users and opioid coprescriptions.
We conducted two post hoc analyses.The first added a three-way interaction between study arm, time, and number of decedents to Model 1 to test whether high-frequency prescribers with multiple decedents were more amenable to repeat comparator letter exposure.The second assessed whether letter efficacy decreased over time by including two fixed interaction terms between the study arm, the study start (weeks 4-22), and the study end (weeks 23-52).We compared study-start and study-end coefficients using an equality of regression coefficients test 19 .Boxplots and corresponding legends were generated using the R officer, cowplot, rvg, and ggplot2 packages 20 .

Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.corresponding author (J.N.D.) through a signed Data Use Agreement.Requests should be submitted to jdoctor@usc.edu;allow 30 days for a response to your request.

Fig. 2 |
Fig. 2 | Clinician-level Opioid Prescribing.Boxplots showing median log MME, the interquartile range, and outliers pre-to-post intervention between study arms among clinicians with one versus multiple decedents.
. The current paper compared a modified letter to the active treatment in Doctor et al. 2018, which resulted in a 12.9% decrease in morphine milligram equivalent prescriptions filled.The new letter appears even more effective than the letter in Doctor et al. (2018).

Fig. 3 |
Fig. 3 | Benzodiazepine Prescriber CONSORT diagram.CONSORT diagram for clinicians who prescribed a benzodiazepine during the study period.

Fig. 4 |
Fig. 4 | Clinician-level Benzodiazepine Prescribing.Boxplots showing median log DME, the interquartile range, and outliers pre-to-post intervention between study arms among clinicians with one versus multiple decedents.

Table 2 |
Prescriber characteristics a Counts of less than 10 were censored in accordance with the State of California Department of Justice's CURES policy.b Two-sample t-test for continuous traits.Chi-square test for categorical variables.

Table 3 |
Adjusted mean total weekly morphine milligram equivalents (MMEs) dispensed between study arms.Values in parentheses are 95% CIs with 9% trimmed means

Table 4
a Predicted DME using coefficients from censored, mixed linear model (TableS1b) testing twosided hypothesis that change in pre-to-post DME does not differ by study arm.